This invention relates to a control system for variably operable devices in response to movement of an object within an area, for example, movement of a person moving within an area of large extent such as a sports playing field or arena or an ice rink or stadium although it could also potentially be used in smaller area such as a theatre or T.V. studio.
1. Background Art
There currently exist a number of 3D position tracking technologies which, operating within a variety of constraints, provide a certain number of updates per second over 3D volumes of varying size. One of these technologies is our GAMS (Gesture and Media System) technology as for example described and claimed in U.S. Pat. No. 5,107,746 issued to Will Bauer on Apr. 28, 1992 and U.S. Pat. No. 5,214,615 issued to Will Bauer on May 25, 1993 and U.S. Pat. No. 5,412,619 issued to Will Bauer on May 2, 1995. There are also other systems based on a variety of measurement methodologies. However, when confronted with the problem of tracking high speed movements over large areas such as ice rinks, sports playing field, etc. none of the systems of these patents or other known 3D tracking technology provide a useful tool for using the position of performers in large areas to control electronic media and other variably operable systems such as virtual electronic sets, robots, computer controllable cameras, and the like. The reasons for this are twofold: the first is that the tracking area of coverage is so large; the second is that performers such as professional skaters may move very quickly. Given the combination of these two difficulties presently available, completely automated system cannot currently follow performers moving a high speeds over large areas. Consider a skater performing on an ice rink. Current show performance practice dictates the use of between one to four xe2x80x9cfollow-spotsxe2x80x9d (spot lights which are operated by a human being and are pointed at the performer as he or she moves on the ice). Any other lighting or sound effect must be played back from a recording and the performer must very carefully time their movements to the music and the lighting areas.
Additionally, the need for such 3D information is not confined to performance situations. Often when a show involving lighting is on tour, its preset cues may be adjusted at each venue due to changes in stage shape. If one can move many lights at once to a new location on the stage (which is entirely possible given such 3D information) and automatically refocus them and resize their beam diameters, a considerable saving of time and money results.
The present inventor has, therefore, addressed the problem in a different manner.
2. Disclosure of the Invention
According to the invention there is provided a control system for variably operable apparatus in response to movement of an object over a large area comprising:
a fixed frame of reference for said large area, comprising at least four calibration points any three of which are not collinear;
a further reference point of origin establishing X, Y and Z coordinates of each of said calibration points, the X, Y and Z coordinates being calculable;
a tracking head, having pan and tilt controls for tracking the movement of an object in the large area and measuring pan and tilt angles of the tracking head for each of said calibration points;
a variably operable system and means to establish control positions of the variably operable system for each of said calibration points and for said point of origin;
computer means to calculate in real time from said X, Y and Z coordinates and from said pan and tilt angles, a series of pan and tilt angles for the variably operable system, a series of the X, Y, Z position and/or the pan, tilt and rotation angle offsets of the tracking head and the variably operable system; and
means to set in real time the variably operable system into conformity with its calculated pan and tilt angles therefor.
Probably the variably operably system is a lighting or music system which may include a number of spot lamps. One or more spot lamps may be operated at a time and a further spotlamp may take over the operation as the skater moves out of range of a first spotlamp. Probably the moving object is a performer such as a skater, dancer or the like.
The invention also includes a method of controlling variably operable apparatus using the system above described.
An example of such control for an xe2x80x9cintelligentxe2x80x9d light involves the use of a high speed serial data communications protocol known as xe2x80x9cDMX-512xe2x80x9d. DMX-512 (xe2x80x9cDMXxe2x80x9d for short) allows a series of lamps to receive commands from a lighting control console, computer, or similarly equipped device. These commands are usually transmitted about 44 times per second and allow control of the pan and tilt angles of the lamp plus brightness, colour, projection pattern (xe2x80x9cgobosxe2x80x9d), and a variety of other parameters. Using a computer equipped with a DMX interface card, it is a simple matter to write programs which direct the light to point at desired pan/tilt angles or modulate any other controllable parameters by sending the appropriate DMX data over the serial data cable to the light(s) in question.
Such control may also be used to make possible the cross-fading of lamps when a performer moves out of range of one lamp and into range of another lamp. The brightness of the first lamp can be faded down as the performer moves away from one lamp and the brightness of the second lamp can be faded up as the performer moves towards it, thus assuring an even distribution of light on the performer at all times regardless of their position in the stage area.
To understand the functioning of the system, consider a pan/tilt tracking head such as that found on a video camera tripod. This subsystem allows a flat plate (which normally holds a camera or some other imaging device but can hold other things such as a telescopic sighting device) to be rotated about two orthogonal axes (normally referred to as the xe2x80x9cpanxe2x80x9d and xe2x80x9ctiltxe2x80x9d axes) by a handle and is thus operable by a human. This tracking head can be pointed in any direction within an almost spherical area, limited only by its mounting methodology. Next, consider a tracking head with shaft encoder or other electronic means to allow measurement of each of the orthogonal pan and tilt angles. From this instrumentation, it is possible to obtain two angular quantities corresponding to the pan and tilt angles at which the head is oriented. These measurements can be obtained sufficiently frequently (up to several thousand times per second, e.g. 100-15000 times per second) as to be considered quasi-continuous for most applications.
If the 3D X,Y,Z location of this tracking head plus any angular pan, tilt, or rotation offsets is known relative to some fixed frame of reference, and if the location of a plane area such as an ice-rink is also known relative to the fixed frame of reference; then it is possible to calculate the distance between the 3D position of the tracking head and a point on the plane area that is collinear with the radial path along which the tracking head is pointed. If the 3D position and angular offsets of the tracking head are known and the pan/tilt angles of where it is pointing are also known, then the coordinates of the point in the plane at which it is pointing can be determined. If the point happens to be, for example, a skater who is moving on an ice rink, as the tracking head is turned to follow the skater, a succession of xe2x80x9cvirtualxe2x80x9d positions are generated giving the skater""s position on the ice in three dimensions (assuming that they remain at a constant height above the ice). Thus the instrumented tracking head acts like a 3D position tracking device attached to the skater, creating a series of virtual 3D measurements of the skater""s trajectory over the ice surface.
In addition to deriving a Z coordinate as mentioned above by projecting the pointing angle of the pan/tilt head and calculating its point of intersection with a plane at a known height Z=Zp, a radial ranging system can be utilized to measure Z directly and obtain the true value of the Z coordinate. To do this, the distance along the pan/tilt head sight line from the pan/tilt head to the performer (or other target) must be measured. One way to achieve this is to use a laser ranging system mounted parallel to the pan/tilt head""s line of sight. Such ranging systems are well known to one skilled in the art and are available with accuracies from several meters to several millimetres.
This virtual positioning information can be used very effectively to control electronic media in response to the performer""s trajectory. It is quite possible xe2x80x9cmapxe2x80x9d 3D positions, velocities and/or accelerations onto electronic media behaviours using computer programs with interfaces both to the position transducing equipment and also to common electronic protocols such as DMX (lighting), MIDI (electronic music and machine control), serial communications, other protocols such as TCP/IP and direct digital or analog electronic interfaces for camera pan/tilt control and the like. Some suitable computer programs have been generally described in U.S. Pat. Nos. 5,107,746, 5,214,615 and 5,412,619 previously referred to. The present invention may use such programs but in conjunction with virtual positioning information from the tracking head rather than positioning information derived from a sensor attached to the performer""s body.
This is of particular use in large-scale situations which are beyond the current limitations of currently available automated 3D tracking systems. Using the system of the invention, the performer can take direct control of the cuing of the media events for the performance, allowing much more interesting effects to be created and utilized. Thus, instead of up to four human follow-spot operators, only one human is required to operate the virtual position tracking head; the response of all follow-spot lights, other lighting, computer controllable cameras, and other media events such as music, sound effects, and video projection are based on the virtual position information derived from the pointing of the tracking head at the performer as he or she moves on the ice.
In order to use the virtual positioning system, there must be means of establishing the 3D location of both the tracking head and any relevant media, such as computer-controllable moving lights, relative to a fixed frame of reference. To accomplish this task, four calibration points are set on the surface of the ice rink or performing surface. These four points must be set out in such a way that any three of them are not collinear. Distances between them and a fifth centre point that defines the origin of the coordinate system can be used to establish their X and Y coordinate positions relative to the centre point which functions as the coordinate origin. Having established these measurements, the tracking head can be pointed at each of the four calibration points and the pan and tilt angle measurements of the tracking head for each calibration point can be obtained. The computer controlled lights and any other appropriate media may be calibrated in a similar manner. Given this information (knowledge of the X,Y,Z coordinates of the four calibration points relative to the centre plus the pan/tilt angles between the tracking head and follow-spot lights and the calibration points) it is possible to calculate both the X,Y,Z position of each light and tracking head plus the light/tracking head""s pan, tilt, rotation angle offsets. Given these six calculated values (X,Y,Z, pan angle offset, tilt angle offset, rotation angle offset) there is complete information to allow the derivation of virtual position information and to allow the lights to be pointed at any virtual points desired.
In addition to pointing the lamps, they can also be automatically focussed and irised based on the virtual positioning information. This ability to auto-iris/focus many lamps on one virtual point is very important for both real-time operation and for pre-show setup where lamp cue positions must be readjusted for each new venue (if a show is on tour).